Skip to content

.Net: Add Provider Support to ONNX Connector (+CUDA Sample)#12861

Merged
markwallace-microsoft merged 11 commits into
microsoft:mainfrom
loicbtd:features/dotnet_onnx_provider_choice
Aug 7, 2025
Merged

.Net: Add Provider Support to ONNX Connector (+CUDA Sample)#12861
markwallace-microsoft merged 11 commits into
microsoft:mainfrom
loicbtd:features/dotnet_onnx_provider_choice

Conversation

@loicbtd

@loicbtd loicbtd commented Aug 5, 2025

Copy link
Copy Markdown
Contributor

Motivation and Context

  1. Why is this change required?
    Currently, users can configure ONNX execution providers by manually creating a genai_config.json file next to the ONNX model.

  2. What problem does it solve?
    This change eliminates the need for manual JSON configuration files by providing a programmatic API for provider configuration, making it easier for developers to configure ONNX execution providers directly through code with the extension methods.

  3. What scenario does it contribute to?
    This improves the developer experience when working with ONNX models in Semantic Kernel, particularly for scenarios where:

    • Developers want to dynamically select providers based on runtime conditions
    • Teams prefer code-based configuration over file-based configuration
  4. If it fixes an open issue, please link to the issue here.
    Closes .Net: Add Provider Choice Support to ONNX Connector #12828

Description

Changes Made:

  • New Provider Class: Added Provider class to encapsulate provider configuration with Id and Options properties
  • Enhanced Chat Completion Service: Extended OnnxRuntimeGenAIChatCompletionService to accept the providers parameter.
  • Updated Builder Extension: Modified OnnxKernelBuilderExtensions.AddOnnxRuntimeGenAIChatCompletion to accept the providers parameter.
  • New Demo Project: Created OnnxWithProviderChoice demo showcasing the new provider configuration API
  • Package Configuration: Add the Microsoft.ML.OnnxRuntime.Gpu package version in the Directory.Packages.props

Design Approach:

The implementation leverages ONNX Runtime GenAI's existing Config API to programmatically set providers and their options. The Provider class provides a clean abstraction for specifying provider ID (e.g., "cuda", "cpu") and custom options. During service initialization, the providers are configured using the underlying ONNX Runtime GenAI configuration system.

Usage Example:

builder.AddOnnxRuntimeGenAIChatCompletion(
    modelId: "onnx",
    modelPath: modelPath,
    providers: [new Provider { Id = "cuda" }]
);

Backward Compatibility:
This change maintains full backward compatibility. Existing code continues to work without modification, and the manual genai_config.json approach remains supported.

Contribution Checklist

@moonbox3 moonbox3 added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel labels Aug 5, 2025
@loicbtd loicbtd marked this pull request as ready for review August 5, 2025 12:23
@loicbtd loicbtd requested a review from a team as a code owner August 5, 2025 12:23
@rogerbarreto

Copy link
Copy Markdown
Member

@loicbtd Thanks for the contribution, I added some extra updates to this PR related to your change and other outstanding changes I had plan myself to add to the connector. Overall LTGM.

@loicbtd

loicbtd commented Aug 6, 2025

Copy link
Copy Markdown
Contributor Author

Thanks for the update 👍

@rogerbarreto rogerbarreto changed the title .Net: Add Provider Choice Support to ONNX Connector .Net: Add Provider Support to ONNX Connector (+CUDA Sample) Aug 6, 2025
Comment thread dotnet/SK-dotnet.slnx Outdated
@markwallace-microsoft markwallace-microsoft added this pull request to the merge queue Aug 7, 2025
Merged via the queue into microsoft:main with commit fe30769 Aug 7, 2025
19 checks passed
ltwlf pushed a commit to ltwlf/semantic-kernel that referenced this pull request Aug 9, 2025
…t#12861)

### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

1. **Why is this change required?**
Currently, users can configure ONNX execution providers by manually
creating a `genai_config.json` file next to the ONNX model.

2. **What problem does it solve?**
This change eliminates the need for manual JSON configuration files by
providing a programmatic API for provider configuration, making it
easier for developers to configure ONNX execution providers directly
through code with the extension methods.

3. **What scenario does it contribute to?**
This improves the developer experience when working with ONNX models in
Semantic Kernel, particularly for scenarios where:
- Developers want to dynamically select providers based on runtime
conditions
   - Teams prefer code-based configuration over file-based configuration

4. **If it fixes an open issue, please link to the issue here.**
   Closes microsoft#12828 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

**Changes Made:**

- **New Provider Class**: Added `Provider` class to encapsulate provider
configuration with `Id` and `Options` properties
- **Enhanced Chat Completion Service**: Extended
`OnnxRuntimeGenAIChatCompletionService` to accept the providers
parameter.
- **Updated Builder Extension**: Modified
`OnnxKernelBuilderExtensions.AddOnnxRuntimeGenAIChatCompletion` to
accept the providers parameter.
- **New Demo Project**: Created `OnnxWithProviderChoice` demo showcasing
the new provider configuration API
- **Package Configuration**: Add the Microsoft.ML.OnnxRuntime.Gpu
package version in the Directory.Packages.props

**Design Approach:**

The implementation leverages ONNX Runtime GenAI's existing `Config` API
to programmatically set providers and their options. The `Provider`
class provides a clean abstraction for specifying provider ID (e.g.,
"cuda", "cpu") and custom options. During service initialization, the
providers are configured using the underlying ONNX Runtime GenAI
configuration system.

**Usage Example:**
```csharp
builder.AddOnnxRuntimeGenAIChatCompletion(
    modelId: "onnx",
    modelPath: modelPath,
    providers: [new Provider { Id = "cuda" }]
);
```

**Backward Compatibility:**
This change maintains full backward compatibility. Existing code
continues to work without modification, and the manual
`genai_config.json` approach remains supported.


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
@loicbtd loicbtd deleted the features/dotnet_onnx_provider_choice branch August 27, 2025 14:37
jcruzmot-te pushed a commit to thousandeyes/aia-semantic-kernel that referenced this pull request Sep 15, 2025
…t#12861)

### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

1. **Why is this change required?**
Currently, users can configure ONNX execution providers by manually
creating a `genai_config.json` file next to the ONNX model.

2. **What problem does it solve?**
This change eliminates the need for manual JSON configuration files by
providing a programmatic API for provider configuration, making it
easier for developers to configure ONNX execution providers directly
through code with the extension methods.

3. **What scenario does it contribute to?**
This improves the developer experience when working with ONNX models in
Semantic Kernel, particularly for scenarios where:
- Developers want to dynamically select providers based on runtime
conditions
   - Teams prefer code-based configuration over file-based configuration

4. **If it fixes an open issue, please link to the issue here.**
   Closes microsoft#12828 

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

**Changes Made:**

- **New Provider Class**: Added `Provider` class to encapsulate provider
configuration with `Id` and `Options` properties
- **Enhanced Chat Completion Service**: Extended
`OnnxRuntimeGenAIChatCompletionService` to accept the providers
parameter.
- **Updated Builder Extension**: Modified
`OnnxKernelBuilderExtensions.AddOnnxRuntimeGenAIChatCompletion` to
accept the providers parameter.
- **New Demo Project**: Created `OnnxWithProviderChoice` demo showcasing
the new provider configuration API
- **Package Configuration**: Add the Microsoft.ML.OnnxRuntime.Gpu
package version in the Directory.Packages.props

**Design Approach:**

The implementation leverages ONNX Runtime GenAI's existing `Config` API
to programmatically set providers and their options. The `Provider`
class provides a clean abstraction for specifying provider ID (e.g.,
"cuda", "cpu") and custom options. During service initialization, the
providers are configured using the underlying ONNX Runtime GenAI
configuration system.

**Usage Example:**
```csharp
builder.AddOnnxRuntimeGenAIChatCompletion(
    modelId: "onnx",
    modelPath: modelPath,
    providers: [new Provider { Id = "cuda" }]
);
```

**Backward Compatibility:**
This change maintains full backward compatibility. Existing code
continues to work without modification, and the manual
`genai_config.json` approach remains supported.


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.Net: Add Provider Choice Support to ONNX Connector

4 participants